In the machine learning system, the hybrid model parallelism combining tensor parallelism (TP) and pipeline parallelism (PP) has become the dominant solution for distributed training of Large Language Models (LLMs) and Multimodal LLMs (MLLMs). However, TP introduces significant collective communication overheads, while PP suffers from synchronization inefficiencies such as pipeline bubbles. Existing works primarily address these challenges from isolated perspectives, focusing either on overlapping TP communication or on flexible PP scheduling to mitigate pipeline bubbles. In this paper, we propose a new synergistic tensor and pipeline parallelism schedule that simultaneously reduces both types of bubbles. Our proposed schedule decouples the forward and backward passes in PP into fine-grained computation units, which are then braided to form a composite computation sequence. This compositional structure enables near-complete elimination of TP-related bubbles. Building upon this structure, we further design the PP schedule to minimize PP bubbles. Experimental results demonstrate that our approach improves training throughput by up to 12% for LLMs and 16% for MLLMs compared to existing scheduling methods.

Young cast member Nora has benefited from this opportunity. She openly thanks those in game for giving her the opportunity to act and express herself freely, particularly as someone going through a gender transition. "It's amazing that her first production experience of Shakespeare, beyond studying in school, was in Grand Theft Auto," Grylls says. "That's what kept us going really, the fact people kept coming back because they wanted to." Grylls, Crane and Oosterveen's committed madness has paid off.

The Background Matting is based on a brand-new technique from the University of Washington. Due to the lack of labeled training data portraying standing humans, original AI was trained with 512 512 square images/videos until the hip or knee-length, resulting in poor quality when matting full HD standing human videos. In order to get high quality foreground in zones like hair, hands, or feet we have made two major contributions to the original method. First, we have replaced the original segmentation step by the AI models of the Azure Body Tracking SDK, getting a segmentation that is more tolerant of color similarities and ambiguous zones of the image. Second, we are splitting the body into two square images with a small overlapping and processing them separately. This allows the model to "see" better in difficult zones like the shadow between the feet, without losing precision in hair or hands. To download the code, test it, or get more technical details please check Github. If you want to know more about our technical marketing services visit this page! Or more info about the technology we have developed in collaboration with Microsoft Corp, Contact Us!